The present development relates to real-time interactive creation of computer graphics images. A variety of computer graphics system peripherals enable the display of electronic imagery. Representative signals are typically raster scanned from a frame buffer memory into the display device. Input is provided by a user through a pointing device. As the user moves the pointing device a trace of the trajectory appears on the display. As described in the Book, Principles of Computer Graphics, Second edition, Newman and Sproull, McGraw-Hill Book Company, 1979, Sections 12-4 and 17-2. Various pointing devices can be used to generate strokes of color in a computer graphics display, just as in painting on a canvas.
Computer based paint applications were developed shortly after the first graphics displays became available. Over time the basic techniques have changed very little. A user specifies a stroke through the use of a locational input device and a line is drawn connecting each input location, or a dot is drawn at each location. These lines or dots were intended to visually represent a pen or brush stroke. Generally, as the terms "paint" and "brush stroke" are used below, they are deemed to embrace painting, inking and so on to accomplish various effects or formats as a brush, airbrush, and so on.
As a stroke is defined using the basic techniques, screen picture elements (pixels) within a specified distance from the input location are colored appropriately. Recently techniques have been refined so that new paint data blends with existing picture information instead of just writing over it. These techniques require tremendous amounts of color and transparency blending processing, putting real-time performance beyond the capabilities of most similar applications. This pixel based approach imposes limitations as to painting techniques, effects, and primitive forms. Such systems are disclosed in U.S. Pat. Nos. 4,602,286 (Kellar, et al.) and 4,633,416 (Walker).
Traditional two dimensional pixel based computer paint systems have been generally embraced by the artistic, design, and entertainment communities. This widespread application comes despite serious limitations with regards to both expense and interactive performance. Current systems are either very expensive due to the special purpose computer hardware required to achieve acceptable performance levels or are slow so as not to accurately reproduce the feel of a physical drawing tool. Inadequate performance is the result of the large amount of processing that is required to color a single pixel in the display device. General purpose processors are unable to process the input at rates necessary to mimic the feel of physical tools.
More recently artists trained in three dimensional modeling and animation techniques wish to paint directly on three dimensional objects rather than simply on a flat canvas. A solution to this requires an object space system which accounts for lighting and other environmental factors. Research along these lines has resulted in a geometric pick and color solution that assigns a new color to the picked polygons and redraws them. Such a solution is restricted to operate along the polygonal borders of the object being painted. All color changes are hard edged and follow the angular edges of the polygonal model. The result appears very unnatural and geometric. A realistic looking result is only attained by generating a secondary output of texture maps that correspond to the "painted" data which can then be used as input to a software renderer. Such a system is described in Computer Graphics, Volume 24, No. 4, August 1990, in an article titled Direct WYSIWYG Painting and Texturing on 3-D Shapes, Hanrahan and Haeberli. The limitations of such a system are prohibitive to the point that there is no known commercial system available based on these techniques.
Prior two dimensional paint systems are based on pixel operations and output while research into three dimensional paint systems is limited to coloring polygons in the existing model structure. This inherent difference between two dimensional and three dimensional techniques is a primary reason for the absence of a widely known, commercially viable three dimensional paint system. True three dimensional painting requires a technique that bridges the gap and combines the appropriate technology from both two dimensional and three dimensional graphics techniques.
The present development takes a new approach to traditional two dimensional painting and provides a viable solution to the three dimensional problem. The approach involves the use of a three dimensional graphic primitive as the basic painting unit rather than pixels. Utilizing such a higher level primitive results in a far more flexible and powerful painting tool. The three dimensional drawing primitive utilizes all the acceleration and functionality of generally available three dimensional computer graphics accelerator devices. Dependence on the host computer processor for performance is eliminated and may be shifted to the graphics accelerator. Virtually any physical drawing tool can be electronically duplicated by using hardware accelerated texture mapping. The basic drawing primitive may also take various forms including a simple polygon, a set of polygons such as a cube or sphere, or any form of polygonal mesh or surface. The polygonal mesh may be subdivided to varying degrees in both the horizontal and vertical dimensions to produce numerous characteristic variations. The use of a three dimensional painting primitive provides a bridge between two dimensional and three dimensional graphics and resolves many of the problems mentioned previously. One method to perform true three dimensional painting is accomplished by holding two dimensions of the polygonal mesh constant and making the third dimension conform to underlying surface information. A typical implementation will hold the X and Y components of the mesh primitive constant while allowing the Z component to be modified to match the underlying surface data at each vertex in the brush mesh. In this sense the three dimensional brush has substantially the same characteristics as the flat canvas brush except that it conforms to the shape of the underlying surface at that location. The three dimensional brush primitive is processed through the accelerated three dimensional graphics pipeline resulting in the visual effect of painting in real-time on either a canvas or some underlying three dimensional surface.